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ABSTRACT 


'Wy*  r- rr;^77-A 


We  prove  that  if  the  second-order  sufficient  condition  and  constraint 
regularity  hold  at  a  local  minimizer  of  a  nonlinear  programming  problem,  then 
for  sufficiently  smooth  perturbations  of  the  constraints  and  objective 
function  the  set  of  local  stationary  points  is  nonempty  and  continuous; 
further,  if  certain  polyhedrality  assumptions  hold  (as  is  usually  the  case  in 
applications)  then  the  local  minimizers,  the  stationary  points  and  the 
multipliers  all  obey  a  type  of  Lipschitz  condition.  Through  the  use  of 
generalized  equations,  these  results  are  obtained  with  a  minimum  of  notational 
complexity. 
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SIGNIFICANCE  AND  EXPLANATION 


In  practical  problems  from  logistics,  structural  design,  chemical 
engineering  and  other  areas,  it  is  often  necessary  to  maximize  or  minimize  a 
nonlinear  function  of  several  variables  subject  to  nonlinear  equation  and/or 
inequality  constraints.  Since  problem  data  or  functional  forms  may  not  be 
known  exactly,  it  is  of  interest  to  know  whether  a  local  solution  to  such  a 
problem  will  persist  under  small  changes  in  the  data  or  in  the  problem 
functions.  In  this  paper,  we  show  that  if  two  fairly  well  known  "niceness" 
conditions  are  satisfied  at  a  local  solution  of  the  unperturbed  problem,  then 
for  small  perturbations  the  perturbed  problem  will  have  one  or  more  solutions 
near  the  original  one.  Moreover,  these  solutions  will  often  display  good 
continuity  properties. 
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GENERALIZED  EQUATIONS  AND  THEIR  SOLUTIONS,  PART  II: 
APPLICATIONS  TO  NONLINEAR  PROGRAMMING 
Stephen  M.  Robinson* 


Introduction .  This  paper  deals  with  the  stability  of  solutions  and  multipliers  of 
nonlinear  programming  problems  when  the  data  of  the  problems  are  subjected  to  small 
perturbations.  The  problem  with  which  we  shall  deal  may  be  formulated  by  introducing 
functions  f  and  g  from  an  open  convex  set  0  C  Rn  to  R  and  tf*  respectively,  a 
closed  convex  set  C  C  R°  and  a  closed  convex  cone  Q  C  if1.  The  problem  of  interest  is 
then 


minimize  f (x) 

subject  to  g(x)  e  Q°  ,  (1.1) 

x  e  C  , 


where  Q°  denotes  the  polar  cone  of  Q: 

Q°  J“  !y  f  r”  I  <q,y >  <  0  for  each  q  e  q)  . 

If  we  assume  that  f  and  g  are  Frechet  differentiable  and  that  certain  regularity 
conditions  are  satisfied,  it  can  be  shown  (171  that  with  each  local  minimizer  x  of  (1.1) 
there  are  associated  one  or  more  multipliers  u  e  I?  such  that  (x,u)  satisfy  the 
necessary  optimality  conditions 


0  e  ,Cx(x,u)  +  ^(x) 

0  e  -X  (x,u)  +  3i^(x)  , 
u  Q 


(1.2) 


where  £'(x,u)  :=  f(x)  +  (u,g(x)>  is  the  standard  Lagrangian,  and  where  the  subscript  x 
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or  u  denotes  partial  differentiation  with  respect  to  that  variable-  The  notation 
denotes  the  normal  cone  at  a  point:  thus. 


C 

3*cOO  ==  \ 


for  a  convex  cone  like  Q, 


{v  £  Rn|(y,c  -  x>  <  0 

0 

the  description  of 


for  each  c  e  c}  if 
if 

3’4»  is  even  simpler: 


X  £  C 
x  t  C  ; 


-Q 


(u) 


{•/  £  |  <  v,u  >  =  0} 

0 


if  u  £  0 
if  u  t  Q  . 


&iy  point  x  which,  with  some  u,  satisfies  (1.2)  is  called  a  stationary  point  of  (1.1); 
there  may  veil  exist  stationary  points  which  are  not  local  minimizers. 

What  we  shall  show  here  is  that  if  xp  is  a  local  minimizer  of  ( *  1)  at  which  f 
and  g  are  twice  continuously  Frechet  differentiable  and  at  which  certain  regularity 
conditions  hold  (specifically,  the  second-order  sufficient  condition  and  constraint 
regularity) ,  and  if  f  and  g  are  smoothly  perturbed,  then  the  set  of  stationary  points 
near  xp,  regarded  as  a  multifunction  (multivalued  function)  of  the  perturbations,  is 
f nonempty  and)  continuous  at  xp.  Further,  we  show  that  the  set  of  multipliers  is  upper 
semicontinuous,  where  these  continuity  properties  are  as  defined  by  Berge  [1).  These 
results  are  proved  in  Section  3,  after  a  review  in  Section  2  of  the  necessary  regularity 
conditions. 

In  Section  4,  we  show  that  if  C  and  o  are  assumed  to  be  polyhedral  (as  will  be  the 
case  in  most  applications),  then  even  stronger  results  can  be  established:  the  distance 
from  a  stationary  point  of  the  perturbed  problem  to  xp,  or  from  an  associated  multiplier 
to  the  set  of  multipliers  associated  with  xn,  obeys  a  kind  of  Lipschitz  Condition- 
Examples  are  given,  both  in  Section  3  and  in  Section  4,  to  show  that  certain  stronger 
statements,  although  plausible,  are  pot  generally  true. 

In  several  parts  of  this  paper  we  use  as  a  device  for  simplifying  and  motivating 
results  the  concept  of  a  generalized  equation.  These  objects,  introduced  in  Part  I  of  this 


paper  119],  are  relations  of  the  form 


0  e  F  f  z )  +  T(z) 


fC  k  if 

where  F  :  R  *  R  and  T  is  a  closed  multifunction  from  K1  to  itself  (often  a  normal- 
cone  operator).  For  example,  if  we  rewrite  (1.2)  in  terms  of  f  and  g,  and  use  the  fact 
that  3'^Cxg(x,u)  =  3s»c(x)  x  3^(u),  we  obtain  the  generalized  equation 

'f’(x)  +  g'(x)*u| 

Of  +  N-^x.u)  ,  (1.3) 

-g(x)  J 

and  this  relation  will  be  used  in  several  ways  in  our  analysis. 

Many  papers  have  already  been  written  about  various  stability  questions  connected  with 

n  k  1  k 

(1.1),  often  in  the  snecial  case  in  which  C  =  R^  and  0  =  R  x  R  (where  R  is  the 

+  + 

non-negative  orthant  in  R*).  This  special  case  formulates  the  standard  nonlinear 
programming  problem 

minimize  ffx) 

subject  to  gi(x)  <  0  (i  =  1,...,k)  (1.4) 

gi(x)  =0  (i  =  k  +  1,...,k  +  t )  . 

We  shall  not  attempt  to  review  all  of  these  papers  here;  rather,  we  mention  only  a  few 
which  illustrate  the  different  types  of  assumptions  that  have  been  imposed. 

Fiacco  and  McCormick  [3,  §§2.4,  5.2]  formulated  a  basic  technique  for  analyzing  (1.4) 
under  the  assumptions  of  strict  complementary  slackness,  linear  independence  of  the 
gradients  of  the  binding  constraints,  and  the  second-order  sufficient  condition.  This 
technique  was  refined,  to  deal  with  general  perturbations,  in  (15]  and  in  [4];  the  basic 
tool,  in  all  three  cases,  was  the  standard  implicit-function  theorem. 

If  the  assumption  of  strict  complementary  slackness  Is  dropped,  then  the  standard 
implicit-f'inction  theorem  can  no  longer  be  used.  However,  if  the  linear  independence 
assumption  and  a  somewhat  strengthened  form  of  the  second-order  sufficient  condition  are 
retained,  then  one  can  still  show  that  the  stationary  point  and  associated  multiplier  are 
(locally)  single-valued  functions  of  the  perturbations,  and  that  they  are  locally 
Lipschitzian  if  appropriate  continuity  assumptions  are  made  on  the  problem  functions.  This 
is  shown  for  the  problem  (1.4)  in  (20],  as  a  by-product  of  a  general  implicit-function 
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theorem  for  generalized  equations.  A  similar  result,  without  the  Lipschitz  continuity,  was 
established  by  Kojima  [10,  Th.  6.4}  as  part  of  his  investigation  of  "strongly  stable” 
solutions  of  (1.4). 

If  one  weakens  the  hypotheses  still  further  by  dropping  the  assumption  of  linear 
independence,  then  the  appropriate  condition  to  assume  in  its  place  is  regularity  of  t‘”e 
constraints  [18}  (see  Section  2;  in  the  case  of  (1.4)  this  is  the  constraint  cualificacien 
of  Kangasarian  and  Fromovitz  [12}).  Kith  this  assumption,  together  with  the  strengthened 
form  of  the  second-order  sufficient  condition  previously  mentioned,  Jtojima  [10,  !h.  7.2} 
has  shown  the  existence  of  a  locally  unique  stationary  point  which  is  continuous  under 
small  perturbations.  Under  similar  hypotheses,  Levitin  [11,  -h .  4}  stated  that  if  the 
minimizer  exists  for  small  perturbations  then  various  properties,  including  Lipschitz 
continuity,  followed.  However,  in  Section  4  below  we  give  an  example  which  appears  to 
satisfy  Levitin's  hypotheses  but  for  which  local  Lipschitz  continuity  does  not  hold. 
Finally,  Sargent  [23]  has  studied  the  existence  ar.d  continuity  of  local  ninimizers  under 
constraint  regularity?  however,  his  methods  are  quite  different  from  those  of  the  other 
papers  mentioned  here,  and  it  appears  that  some  of  the  results  in  [23]  may  not  be 
completely  correct. 

We  begin  our  analysis,  in  the  next  section,  by  reviewing  a  generalized  form  of  the 
well  known  second-order  sufficient  condition,  and  exploring  some  of  its  properties. 

2.  a  review  of  the  second-order  sufficient  condition.  The  second-order  sufficient 
condition  is  a  very  well  known  regularity  condition  in  nonlinear  programing.  It  is 
discussed  for  the  standard  nonlinear  programming  problem  (1,4)  in  [3,  52.3};  versions 
adapted  to  problems  in  more  general  spaces  are  given  by,  e.g.,  Guignard  [61,  Maurer  and 
2owe  [13]  and  Maurer  [14}. 

In  this  section  we  exhibit  a  form  of  this  condition  suitable  for  the  problem  (1.1). 
although  we  have  not  seen  this  particular  form  in  the  literature,  it  is  not  likely  to  be 
very  surprising  to  anyone  familiar  with  the  field,  what  may  be  less  familiar,  however,  is 
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its  motivation  in  terns  of  generalized  equations  and 
like  Theorem  2.3. 


its  use  to  prove  isolation  results 


TO  develop  a  second-order  sufficient  condition  for  (1.1),  we  return  to  the  generalized 
equation  (1.3)  which  formulates  the  necessary  optimality  conditions.  We  have  shown  in 
[19,20]  that  important  aspects  of  the  behavior  of  a  generalized  equation  are  captured  in 
its  linearization  about  a  given  point.  To  make  use  of  this  linearization  in  the  present 
case,  we  assume  that  (Xg,ug)  is  a  point  satisfying  (1.2);  the  linearized  form  of  (1.2) 

(or  equivalently,  of  (1.3))  at  (x0,ua)  is  then 


*  sWx,u)  • 


'  WV  ‘ 

■  Xxx(x0'u0) 

£xu{W 

"  ‘ ""  1 

0  f 

.  --WV  . 

4- 

_-£ux(X0'V 

-£uu(W  . 

o 

3 

1 

Simplifying  this  expression,  we  obtain 


'  WV  ' 

■£xx<X0'V  9,(VT' 

*  "  xo 

0  € 

-9<x  > 

L  ^ 

-9Mx0)  0 

u  -  u„_ 

+  3WW  • 


Examination  of  the  form  of  (2.1)  leads  us  to  the  realization  that  it  formulates  the 
necessary  optimality  conditions  for  a  certain  quadratic  programming  problem,  namely 


(2.D 


minimize 


f(x0)(x  -  V  +  j  <X  ‘  V£xx:W(x  *  X05> 


subject  to  g(Xg)  +  g'(xQ)(x  -  xQ)  e  q=  ,  x  f  C  . 


The  quadratic  programming  problem  (2.2)  is  not  derived  by  any  straightforward  approximation 
of  the  functions  in  (1.1)  (note  that  the  second-order  term  in  the  objective  function 
contains,  not  f"(x0),  but  Xxx(x0,Uq) ) .  However,  it  has  been  found  to  be  an  appropriate 
approximation  of  (1.1)  for  nveaerical  purposes  [9,251.  Since  the  derivation  through 
linearization  of  (1.2)  is  straightforward,  we  shall  refer  to  (2.2)  as  the  linearization,  or 
linearized  form,  of  (1.1). 

if  we  are  looking  for  a  condition  to  impose  on  a  stationary  point  (XgjUj,)  of  (i.l) 
to  ensure  that  Xq  will  be  a  strict  local  mininizer  of  (1.1),  an  obvious  approach  to  take 
is  to  impose  some  such  conditions  on  (2.2).  In  fact,  the  simplest  way  of  obtaining  a 
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at  u„,  u.  also  beloncs  to 


show  that  c  >  p.  Sfe  can  suppose  without  loss  of  generality  that  { (x.  -  x  J/Ix  -  x 

i  0  1  u 

converges  to  sooe  h  c  nT‘;  evidently  Ihl  =  1  and  h  e  ?c( xft } .  We  have  for  each  i 
Q*  i  =  g(xQ)  *  gMx  )<x  -  x  1  +  o(Ix.  -  x  I)  , 


g*(x0)h  -  lin  lx.  -  x  I  [c{x,>  -  gcx  )1  t  T^fgfx  » 


have  previously  resarked  that  in  such  a  case  f’lx.lh  >  0,  but  since  for  each  i 


1  II  IP  il1  ill'll  Uiill  il .ilukilii'tl ilhltlii >1  '1>i ! . . 


Subtracting  f(xQ),  dividing  by  lx.  -  xnI^  and  passing  to  the  iir.it ,  w»  find  that 

C  =  {h'ixx(x0'u0,h>  ^  P,b|2  ' 

ar.d  the  theorem  now  follows  by  contraposition. 

Tr.eoren  2.2  shows  that  a  stationary  point  of  (1.1)  satisfying  the  second-order 
sufficient  condition  is  a  strict  local  ninisizer.  A  reasonable  question  to  ask  is  vhethe 
it  is  also  an  isolated  local  ainicizeri  that  is,  whether  there  is  sene  neighborhood  of 
x0  containing  no  other  local  ssiniaizer  of  (1.1).  As  we  shall  now  see,  this  is  not  so. 
even  for  very  single  problems.  Consider,  for  instance,  the  problem 

...  12 

=ini=ize  —  x 

subject  to  x6  sin(1/x)  *  0  .  [sin(l/0>  tm  0.} 

Here  C  =  R  =  Q;  the  feasible  region  is  {0}  U  {(ns)  *  in  =  ±1,-2,...}.  The  seeond-ordes 
sufficient  condition  is  satisfied  at  the  origin;  however,  the  origin  is  a  cluster  point  © 
the  feasible  region  and  ever;  feasible  point  is  a  local  ainiaizer. 

Cue  night  reply  that  such  bad  behavior  is  not  very  surprising  since  (2.51  is  a  bad 
problea.  This  is  quite  true,  and  we  shall  see  that  a  result  of  the  type  we  see's  can  ince 
be  proved  if  we  exclude  bad  probless;  that  is,  if  we  in  pose  a  constraint  qualification. 
Ihe  qualification  we  shall  use  is  the  requirement  that  the  constraints 

g(x)  t  Q- 
x  e  C 
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.211  Cpossibly 


sossoose  thJ 


ssf£iciest  condition  is  at  Cre,3 g)  seith  sone  positive  sodeles.  Uses  there 

exists  a  setritorlaos  if  of  scch  that  if  fx.sj  satisfies  Cl. 21  aas  r  s  V  then 

X  =  XQ. 

PROOF:  By  contraposition.  te?osc  that  there  Is  a  setter see  tCx.,tt5lJ  sach  that 
tx_}  cssterges  to  xg,  and  sseh  that  for  each  ts,  *,  -  »,  and  »i"ss 

«sssr=e  vittart  loss  of  Generality  that  {tx.  -  xr  '•  /lx.  -  xB*i  ccmrerces  to  h. 


Ssstse  vstMK  loss  of  ceneraitty 


.oss  trees  v 


F  =  ipj 


isritcaSit  Kir 


of  the  rrrocf  of  Tbeoreta  3.2  rsahes  ao  sse  of  the  present 


any  result  depending  on  it) .  sccordingly,  sd.th  no  loss  of  central 


sonoose  that 


converges  to 


h.  t -  ) 


lx-.tBJ  Sitisr? 


continuity  an : 


losedtess  of 


•31  sfith  (X|  >S;3 


Ha*  froa  '1.1)  see  find  that  for  each  £,  =C x. J 


x-  r  C;  reasons: 


estaM ;r,n  that  h  < 


fMx^lh  >  0. 


lx, J  satisfies 


,gfx_}i 


.31  for  each  i,  see  hare 

,;’x, )’  +  <n_,g'(x„JCx,  —  r,W  +  oflx.  -  x. 


s*ere  see 


used  the  fact  that  the  oj  are  tail 


and  g(xQ)  €  Q°  we  have  {u^,g(Xg)>  <  0  and  therefore 

0  <  <  ui  ,g'  (x(j)(xi  -  xf)>  +  odXj^  -  x0l) 
Dividing  by  lx^  -  x^S  and  passing  to  the  limit,  we  obtain 

<  Ug  ,g'  (Xg )h)  >  0  . 

Now  from  (1.3)  we  again  have  for  each  i 

f'(xi)  +  g’(x.)*u.  c  -  3^ctxi)  , 


0  >  f' (xi>(xi  -  xQ)  +  <  ,g'  (xi )  (xi  -  xQ)>  , 

and  upon  division  by  Sx^  -  xQ3  and  passage  to  the  limit  we  find,  using  (2.7),  that 
f'(x0)h  <  0  and  therefore  that  h  satisfies  the  hypothesis  of  the  second-order 
sufficient  condition. 

We  know  that 

^C*Xi*  3  -jCx*Xi,Ui*  =  “Kx^xo »u< )  “  £Xx<xo'ui^xi  "  x0>  +  “  x0|)  <  (2.8) 


3W  3  _£X(X0'U0)  ’ 


(uj  3  gtx^  =  g(xQ)  +  g,(x{))(xi  -  xQ)  +  g"(x0)(xi  -  x0>  +  o(Ixi  -  xQ«  ),  (2.10) 


9'I'q<u0>  3  <*(*„>  . 

Using  ( 2.8 )— ( 2.11)  together  with  the  monotonicity  of  3ij»  and  30.  we  find  that 


(2.11) 


0  <  <-g'(x0)*(ui  -  uQ>  -  ^VV^i  ~  *0>'Xi  -  x0>  +  °(«xi  ~  V  >  '  (2*12> 


0  <  < ui  -  ugf9'(x0)(xi  -  xg)  +  g"(xg)<xi  _  xg)2  +  o(lxi  -  XgS2))  •  (2.13) 

Adding  (2.12)  and  (2.13),  we  obtain 

2 

0  <  ("£xx(x0'u0>(xi  “  xo),Xi  -  V  +  o(!xi  “  xo!  *  ' 

2 

and  division  by  lx^  -  x^i  and  passage  to  the  limit  yields  a  contradiction  to  the  second- 

order  sufficient  condition.  This  completes  the  proof. 

We  have  thus  shown  that  when  regularity  is  added  to  the  second-order  sufficient 


condition,  the  stationary  points  xn  which,  with  some  un,  satisfy  (1.2),  are  isolated. 
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It  is  an  easy  consequence  of  this  tact  that  if  the  conditions  of  Theorem  2.3  hold  in  some 
compact  region  of  ,  that  region  contains  only  finitely  many  stationary  points  (perhaps 
.none).  However,  this  result  says  nothing  about  the  behavior  of  stationary  points,  or 
minimizers,  of  problems  which  are  "near”  (1.1)  in  the  sense  that  their  problem  functions 
are  slight  perturbations  of  those  of  (1.1).  In  the, next  section,  we  use  the  second-order 
sufficient  condition  and  the  regularity  condition  to  analyze  the  behavior  of  such  problems. 


3.  Local  solvability  of  perturbed  nonlinear  programming ^problems.  In  this  section  we 
shall  be  concerned  with  the  problem 


minimize  f(x,p) 
subject  to  g(x,p)  e 


x  e  C  , 


where  p  is  a  perturbation  parameter  belonging  to  a  topological  space  P,  and  x  is  the 
variable  in  which  the  minimization  is  done.  The  functions  f  and  g  are  defined  from 
(2  x  P  to  R  and  Kn  respectively,  where  11  is  an  open  set  in  jf' ;  Q  and  C  are  as 
previously.  We  shall  identify  (1.1)  with  the  particular  case  of  (3.1)  arising  when  p  is 
some  fixed  pQ  c  P;  our  interest  here  will  be  in  predicting,  from  information  in  (1.1), 
aspects  of  the  behavior  of  (3.1)  when  p  varies  near  pg,  such  as  solvability,  location 
of  minimizers,  etc.  Throughout  Sections  3  and  4  we  shall  make  the  blanket  jssumption  that 
f(*,p)  and  g( •  ,p)  are  Frechet  differentiable  on  SI,  that  f,  g,  f’  and  a*  are 
continuous  on  SI  x  p,  and  that  f(-,p0)  and  g(*,Og)  are  twice  Frechet  differentiable 
at  Xg,  a  point  in  C  ■n  0  which  is  a  stationary  point  of  (3.1)  for  p  =  pg.  The 
stationary-point  conditions  for  (3.1)  are 

0  c  f * ( x , p)  +  g’(x,p)*u  +  3^c(x) 

(3.2) 

0  c  -g(x,p)  +  3tg(u)  , 


and  we  define  U0  :=  {u  e  R‘  I  (xQ,u)  satisfies  (3.2)  for  p  =  pQ) .  We  shall  denote  the 


distance  from  a  point  a  to  a  set  A  by  d[a,A}  :=  inflSa  -  a' I  !  a'  e  A)  C-“ 
if  A  =  d) . 

Our  first  theorem  shows  that  if  the  second-order  sufficient  condition  holds  at  a  local 
minimizer  at  which  the  constraints  are  regular,  then  that  local  minimizer  persists  under 
small  perturbations. 

THEOREM  3.1:  Suppose  that  for  p  =  pQ,  (3.1)  satisfies  the  second-order, sufficient 
condition  at  Xg  and  some  Ug  e  Ug  and  its  constraints  are  regular  at  Xg. 

Then  for  each  neighborhood  M  xQ  I)  there  is  a  neighborhood  S  of_  pA  such 

that  if  p  e  K  then  (3.1)  has  a  local  minimizer  in  M. 

PROOF:  By  hypothesis,  for  p  =  p0  the  constraints  of  (3.1)  are  regular  at  Xg.  3y 
(18,  Th.  1]  there  are  neighborhoods  Mg  and  Ng  of  xQ  and  pQ  respectively,  and  a 
constant  ? ,  such  that  for  each  x  e  C  A  mq  and  p£  Ng , 

d[x,F(p)]  <  ;d(g(x,p) ,Q“)  , 

where  F(p)  :=  !*e  C  I  g(x,p)  e  Q°}.  Let  eQ  be  the  modulus  for  the  second-order 
sufficient  condition  at  (Xg,ug);  let  e  e  (0,eQ)  and  choose  a  positive  6  so  that 
(xQ  +  26B)  C  Mg  n  m  n  v,  where  V  is  the  neighborhood  of  xQ  given  by  Theorem  2.2  for 
the  chosen  e.  Select  a  neighborhood  of  pg  with  Ng  C  nq,  and  some  positive  u, 

such  that  if  p  t  Sj  and  if  x1 ,x2  «  xQ  +  26B  with  Ix^  -  x2>  <  a,  then 

Iffx^.p)  -  f(x2,Pg)I  <  e62/16  =:  B.  Next,  find  a  neighborhood  N  of  Pg  with  N  c  Ng 
and  such  that  if  p  e  N  then 

C  sup{ Ig(x,p)  -  g(x,Pg)S  |  x  e  xQ  +  6b}  <  min{a,5/2>  . 

Denote  f(xg,p0)  by 

Now  choose  any  p  e  N.  If  F(p)  A  {x  |  lx  -  xQl  =  6}  is  not  empty,  let  x'  be  any 

point  in  it.  One  has  x'  e  C  A  +  6b}  and  g(x',p)  e  Q°,  so 

d(x',F(p0))  <  Cd[g(x’ ,Pg) ,Q°]  <  ?lg(x',Pg)  -  g(x',p)l 

<  min {o,6/2}  . 

Thus  there  is  some  x'  £  F(p„)  with  lx'  -  xll  <  min{a,6/2} ,  and  therefore  with 
0  0  0  a 

IXg  -  xQl  >  lx'  -  Xgl  -  lx'  -  Xgl  >  6/2.  Accordingly,  by  Theorem  2.2  we  have 


f<X0,P0)  =  ^0  +  I  e(6/2>2  =  Po  +  2e'  Als°’  lx0  ~  x0*  =  ,x'  ~  '■)’  + 

"x'  -  x6'  i  ^  +  =  »  so  Xq  e  xQ  +  25b;  as  ix^  -  x*  I  <  a  we  have 

if(x',p)  -  ftx^pp)!  <  8-  Therefore 


f(x',p)  >  f ( Xq , pQ )  -  lf(x',p)  -  f(x^,p0)l 


>  $0  +  28  -  8  =  $Q  +  S. 


It  is  also  true  that 


<3 1  JtQ  ,F(  p)  ]  <  ?d[g(x0  ,p)  ,Q°]  <  Clg{x0,p)  -  g(xc,p0>l 
<  min {a, 5/2}  . 


accordingly,  there  is  some  x"  e  F(p)  with  IxQ  -  xM!  <  min{a,S/2}.  Thus 
lf(x",p)  -  f<x0#p0)S  <  0,  so 

f(xH,p)  <  f(x0,p0>  +  5f(x",p)  -  f(x0,P())8  <  (J>Q  +  3  .  {3.4] 

Putting  (3.3)  and  (3.4)  together,  we  see  that  a  minimizer  of  the  function  f(»,p)  on  the 
set  F(p)  A  ( Xq  +  6B)  exists  (by  compactness  and  the  fact  that  x"  belongs  to 
F(p)  A  (xQ  +  5b)),  but  that  no  such  minimizer  lies  on  F(p)  fi  {s  |  lx  -  xQl  *»  5). 
Therefore  each  such  minimizer  is  in  fact  a  local  minimizer  of  f(*,p)  on  F(p):  that  is, 
a  local  minimizer  of  (3.1)  which  belongs  to  M.  This  completes  the  proof. 

Our  next  results  will  concern  the  continuity  behavior  of  local  minimizers  and  of 
stationary  points  of  (3.1).  For  ease  of  reference,  we  proceed  now  to  define  certain 
multifunctions  which  display  these  points.  Assume  that  the  hypotheses  of  Theorem  3.1  hold 
for  (3.1)  with  p  «=  p0;  then  it  follows  from  the  results  of  [16]  that  there  are 
neighborhoods  M1  of  Xq  and  of  pQ  such  that  for  any  p  e  and  any  x  f  M1 

satisfying 


g(x,p)  <r  Q° 


x  e  C  , 


the  system  (3.5)  is  regular  at  x. 
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At  this  point  it  will  be  useful  to  note  a  property  of  the  multipliers  which  we  shall 
shortly  use.  we  shall  show  that  if  regularity  holds  at  Xg  for  p  -  pg,  then  the 
multipliers  in  (3.2)  are  uniformly  bounded,  and  in  fact  the  set  of  all  such  multipliers  is 
an  upper  semiccntinuous  multifunction  of  (x,p).  This  extends  a  result  in  [11],  also  give 
in  [5]. 

THEOREM  3.2:  If  the  system  (3.5)  is  regular  at  Xg  for  p  =  pc,  then  there  exist 

neighborhoods  M2  of_  xQ  and  h'2  £f  P0,  such  that  if  U  :  x  Rn  and 

SP  :  N„  *  M.  are  multifunctions  defined  by 
2  2  ■  — — —  .....  ■  -  ■>. 

U{x,p!  :=  tu  e  Rr  |  (x,u,p)  satisfies  (3.2)}  for  (x,d)  e  x  n2  , 

SPtp)  :=  {x  e  V.  |  for  some  u,(x,u,p)  satisfies  (3.2)}  for  p  «  Kj  , 

then  a  and  SP  are  upper  semicontinuous. 

PROOF:  We  first  show  that  U  is  locally  bounded  at  (Xg,pg).  Assume,  on  the 

contrary,  that  there  are  sequences  {xj  C  £1  and  {p^}  C  P  converging  to  xQ  and  pQ 

respectively,  and  a  sequence  {u  }  C  Rn  with  lim  Eu. I  =  +“,  such  that  for  each  i  the 

1  i-«» 

triple  (x^u^p,)  satisfies  (3.2).  With  no  loss  of  generality  we  can  suppose  that 
u./S'^J  converges  to  seme  y.  For  each  i,  (3.2)  irsolies  that  ui  e  f>  (a  cone) 
and  ( u^ ,g(x^ ,p^)>  =  0;  dividing  by  lu^  and  passing  to  the  limit  we  find  that 

y  f  Q  and  <y,g(x0,pg)>  =  0.  Again  from  (3.2),  we  have  for  each  i, 

f*(x  ,p,)  +  g' (x<fp.  )*u.  e  -S'l^fx^)  ; 

again  dividing  by  !u^»  and  passing  to  the  limit,  using  the  fact  that  3yc  is  a  closed 
multifunction  whose  values  are  cones,  we  obtain  (Xg,Pg)*y  6  -a^fXg).  By  regularity, 
for  some  e  >  0  there  is  a  point  x  e  C  with  g{x„,r>_)  +  g'(x„,pn)(x  -  x„)  +  ey  e  Q°. 

Using  our  information  about  y,  we  find  that 

0  >  (  y,C(  Xg  ,Pg  )  +  g'  (Xg,Pg)(Xr  “  Xg)  ♦  ^  ^  ) 

2 

>  C»yl  >  0  , 


a  contradiction 


It  follows  that  there  rust  exist  neighborhoods  m2  of  Xg 


in  £i  and 


N,  of  p0  in  P,  and  a  compact  set  K  C  Rp,  such  that  if  (x,p) f  x  ko  then 
U(x,p)  C  K.  Without  loss  of  generality  we  can  suppose  that  M,  and  S,  are  small  enough 
so  that  ^2  is  bounded  and  that  for  (x,p)  e  M}  x  the  values  lg(x,p)l,  if’(x,p)l 
and  lq’(x,p)l  all  satisfy  some  uniform  bound. 

Now  let 


h(x,p,u) 


f ' !x,p)  +  g1 (x,p)*u 
-g(x,p) 


we  note  that  there  is  some  compact  set  L  such  that  if  (x,p,u)  e  m  x  s2  x  k  then 
h(x,p,u)  e  L. 

Now  denote  by  G  the  graph  of  the  multifunction  3#c  x  3^,  and  define 

H  :=  {(x,p,u,v)  |  (x,p)  e  M2  x  S’2,  (x,p,u,v)  e  graph  h,  (x,u,-v)  e  g}  . 

The  continuity  assumptions  and  the  properties  of  3^  and  3$g  imply  that  H  is  closed 
in  M2  x  n2  x  r1  x  Rn+ra.  However,  if  (x,p,u,v)  e  H  then  (x,p)  e  M2  x  }J2  and  (x,u,p) 
satisfies  (3.2),  so  that  u  e  K  and  therefore  v  e  L.  As  K  and  L  are  compact  it 
follows  that  the  projection  of  H  on  the  space  of  the  first  three,  or  of  the  first  two, 
components  of  (x,p,u,v)  is  closed.  The  first  of  these  projections  is  the  graph  of  the 
multifunction  D,  and  the  second  is  that  of  SP.  Thus  U  and  SP  are  closed;  however, 
as  the  image  of  U  is  contained  in  the  compact  set  K  and  that  of  SP  in  the  precompact 
set  M2,  ;t  follows  that  U  and  SP  are  actually  upper  semicontinuous.  This  completes 
the  proof . 

Our  main  result  on  continuity  of  local  minimizers  and  stationary  points  is  given  in 
the  next  theorem.  For  the  multifunction  SP  and  the  set  Mj  in  the  theorem,  SP  n  M3 
denotes  the  multifunction  defined  by  (SP^1  M^Mp)  ;=  SP{p)  n  Mg. 

THEOREM  3.3;  Suppose  that  for  p  =  Dq,  for  sone  f  S  and  each  u  e  U(xq,Pq), 
(3.1)  satisfies  the  second-order  sufficient  condition  at  !x0,u),  and  that  its  constraints 
are  regular  there. 
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Then  there  exist  neighborhoods 


"3  Of  and  K,  of_  such  that  if  the 

multifunction  LM  :  is  defined  by 

LM(p)  :=  {x  !,  |x  is  a  local  minimizer  of  (3.1)}  ,  'l.*--' 

then  SP  n  Mj  is  continuous  p  and  for  each  p  *  K,  one  has  d  *  LMf  p)  -  SP<pi. 

PROOF:  Let  m2  and  w  be  the  neighborhoods  of  xn  produced  by  Theorems  3.2  a--5  T-* 
respectively;  define  Mj  :=  M2  O  K;  suppose  in  additit.i  that  M-  is  bounded  and  is  small 
enough  so  that  for  some  neighborhood  N,  of  p0,  ary  p  e  S,  and  any  x  «  ",  satisfyi-c 
(3.5),  the  system  (3.5)  is  regular  at  x.  Let  ;.‘2  be  the  neighborhood  of  pr  given  by 
Theorem  3.2;  let  Ne  be  the  neighborhood  obtained  by  applying  Theorem  3.i  to  (3.1)  with 
M  =  M 3,  and  define  N3  :=  N,  n  n  s_.  Define  LM  as  in  C3.6). 

Me  shall  first  show  that  LM  and  SP  O  Mj  are  lower  semicontinuous  at  pQ.  By 
Theorem  2.3,  SP(pQ)  *3  M,  =  {x^} ;  also,  for  any  p  e  113  if  x  f  LM(p)  then  since 
regularity  of  (3.5)  is  a  sufficient  constraint  qualification  -17]  we  have  x  t  SPsp)  ~  M  . 
Thus  if  p  e  Nj  then  <5  *  LM(p)  C  SP!p)  •'">  M^.  However,  since  xp  e  LM(pp)  by  Theorem 
2.2,  we  actually  have  LM(p  )  =  SP(pn)  ^  Mj  =  ixQ}.  Let  S  be  any  open,  set  in  M,  with 
S  O  LM(Pp)  *  6;  then  obviously  (S?  MjHpg)  C  S.  Sow  by  applying  Theorem  3.1  to  S  we 

can  find  some  neighborhood  Ne  of  pn  such  that  NV  C  k3  and  if  p  «  Kg  then 
LM(p)  Oslo.  Hence  LM  is  lower  semicontinuous  at  p„,  but  since  LM  C  SP  K, 
and  LM(Pq)  =  (SP  n  M3)(p0),  we  see  that  SP  n  M3  is  also  lower  ser.icor.tinucus  at  pp. 

But  SP  is  upper  semicontinuous  from  N,  to  M 2,  so  SP  M-3  is  upper  semicontinuous 
from  N3  to  m3,  and  thus  SP  .M-,  is  actually  continuous  at  pp.  This  completes  the 
proof. 

Theorem  3.3  shows  that  that  portion  of  the  set  of  stationary  points  near  x-  is 
continuous  at  pp,  but  it  gives  no  measure  of  how  it  depends  on  p.  In  the  next  section, 
we  show  that  if  the  sets  C  and  Q  are  polyhedral  then  the  dependence  of  SP  and  0 
upon  p  can  be  measured  by  the  dependence  of  the  problem  functions  and  their  derivatives 
upon  p:  in  particular,  if  the  latter  are  Lipschitzian  then  S?  is  "upper 
I.ipschitzian”.  in  the  meantime,  however,  we  present  an  example,  taker,  free.  ;20]  ,  to  show 


that  LM  say  not  be  single-valued  near  pg  and  that  the  inclusion  of  LM  in  S?  say  be 
strict. 

Consider  the  quadratic  programming  problem 

12  2 

minimize  —  (x^  -  )  -  nx1 

subject  to  _5Ci  +  2x2  -  ® 

_X1  ‘  2x2  1  0  ' 

where  n  is  the  parameter.  For  n  =  0  this  problem  has  a  unique  mininizer  at  the  origin; 

2 

for  any  positive  n  it  has  local  minimizers  at  —  n(2,±1)  and  a  saddle  point  at  (r.,0). 
Thus  in  this  case  LM  is  multivalued  and  is  strictly  contained  in  SP  whenever  n  >  0. 

4.  Upper  Lipschitz  continuity  in  the  .polyhedral  case.  Throughout  this  section  we 
shall  assume  that  C  and  Q  are  polyhedral  convex  sets,  and  we  shall  see  that  in  this 
case  we  can  prove  stronger  results  than  we  did  in  Section  3.  Specifically,  we  shall  prove 
that  the  solutions  and  multipliers  of  (3.1),  regarded  as  multifunctions  of  p,  are  locally 
upper  Lipschitzian  at  pQ  under  appropriate  continuity  assumptions  on  the  problem 
functions.  We  recall  that  if  F  is  a  multifunction  from  F P  to  if1,  F  is  said  to  be 
locally  upper  Lipschitzian  at  zQ  e  tp  if  there  are  a  neighborhood  Z  of  Zg  and  a 
constant  such  that  for  each  z  f  Z, 

F(z)  C  F(zq)  +  Xlz  -  zqIB  , 

where  8  is  the  unit  ball  in  iP.  We  showed  in  [21]  that  any  polyhedral  multifunction 
(one  whose  graph  is  the  union  of  finitely  many  polyhedral  convex  sets)  is  locally  upper 
Lipschitzian  everywhere  with  a  uniform  constant. 

We  begin  by  formulating  a  general  continuity  result  for  nonlinear  generalized 
equations.  It  says  .that  if  a  certain  condition  is  satisfied  by  the  linearizations  of  the 
nonlinear  problem,  then  the  latter  has,  at  least  locally,  an  upper  Lipschitzian  inverse 
(which,  of  course,  may  not  be  single-valued  and  might  even  be  empty).  As  in  Section  3,  for 
a  set  S  and  a  multifunction  F,  we  define  the  multifunction  F  n  S  by 
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it 


! F  i  S)!x)  :=  rCx)  "*  S.  The  norm  on  a  product;  space  is  taken  to  be  the  maximum  of  the 
norms  of  the  component  spaces. 

THS08SX  4.1:  Let  F  be  a  Frechet  differer.tiabl  »  function  from  an  open  set  H  C  R" 
to  R“.  Let  R  be  any  polyhedral  multifunction  from  Rn  to  Rn,  and  define  H  :=  F  * 
For  Xp  £  H  and  x  e  Rp  define 

LF  <x)  :=  F< x  !  *  F '  { X_  )  ( X  -  x.)  . 

Xp  0  0  0 


Suppose  there  is  some  compact  set  XQ  C  H  such  that 


^_)  For  each  x«  and  x1  _i£  Xq, 
the  sane. 


the  restrictions  of  LF. 


and  LFX  to  Xq  are 


ii)  For  some  positive  y  and  each  xg  £  Xg, 

X  Fi  (LF  +  R)-1  (0) 

Y  xo 


where  X^,  :=  XQ  *  y3. 

iii)  F'  is  continuous  on  Xr,  and  there  is  a  function  9  :  R  +  R.  with 

■ 1  ■  - - -  “  -  ■  v  ■  1  +  +  -  ~  1 

lim  9(t)  =  o  =  8(0),  such  that  for  each  x  e  X  and  each  xQ  £  xn, 

t+0  Y 

IF(x)  -  LF  <x)i  <  lx  -  x„36(lx  -  x„I). 

x  =  0  0 

Then  there  is  some  positive  S  such  that  the  multifunction  H  ^  is  locally 

upper  Lipschitzian  at  0,  with  (H  1  F!  x.HO)  =  X_. 

*  *  *  *  *  •  — - —  -  o  o 

We  remark  that  (iii)  is  satisfied  in  particular  if  0  is  convex  and  F'  is 
Lipschitzian  there. 


PROOF:  For  x„  £  Xn,  define  T  to  be  (LF  +  R)-1.  Let  e  :=  r:  Y.  We  first 


1 


show  that  there  exist  a  constant  A  and  some  positive  n  such  that  for  any  Xg  e  Xg, 

T  Fi  x  is  upper  Lipschitzian  on  nB  with  modulus  A.  Choose  any  xQ  £  Xg?  then  the 
X0 

sun  LF  +  R  is  polyhedral  and  by  (21,  Prop.  1]  there  are  a  constant  A(x„)  and  a 
x0  0 

positive  nix.)  such  that  T  is  upper  Lipschitzian  on  n(x  )B  with  modulus  A(x  ).  If 

y  Xq  0  0 

we  take  n(x„)  small  enough,  it  follows  from  the  fact  that  (T  F»  X  HO)  =  X„  that  the 

0  Xg  e  0 

multifunction  T  n  x  is  upper  Lipschitzian  on  n(x  )B  with  modulus  A(x  ).  Now,  using 
xQ  e  o  0 

the  continuity  assumptions#  choose  a  neighborhood  Nn  of  xn  small  enough  that  for  any 


X  £  X„  FI  J5 
0  0 
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a)  X(xQ)iF,(x)  -  F’(x0)S  <  -  , 

1 

b)  For  any  x1  e  X  ,  ILF  (x‘)  -  LF  (x’)i  <  -  n(x  ) 


Choose  any 


ny  x  e  X0  i:Q,  and  any  q  f  ^  r,(xn)B.  If  w  e  (T^  X£)(q)  then  w  e  X£  and 


and  thus 


However, 


a  £  LF  (w)  +  R(w)  , 
x 


q  +  LF  (w)  -  LF  (w)  f  LF  (w)  +  R(w) 

xo  51  xo 


W  £  (T  n  X  ) Iq  +  LF  (w)  -  LF  {w)J  . 

X(1  E  xo 


!q  +  LF  (w)  -  LF  (w) 5  <  Iql  +  SLF  (w)  -  LF  (w) I 
x0  x  -  x0 

<  ^  n(xQ)  +  \  n(xQ)  =  n(x0)  , 


so  by  the  upper  Lipschitz  continuity  we  have 


w  £  X„  +  X  ( x  )  S  q  +  LF  (w)  -  LF  {w)IB  . 
0  0  x 


Now  let  x^  be  a  point  of  XQ  with  d[w,Xg]  =  Sw  -  x^I.  By  hypothesis,  since  x0,  x 
and  xn  are  in  XQ,  we  have  LFX  (x,)  =  LFxtx1).  Therefore 


SLF  (w)  -  LF  (w)3  =  SLF  (x,  )  -  LF  (x.)  +  [F*{xn)  -  F*(x)](w  -  x  )I 
X  X  X-  1  X  1  0  I 

0  0 


and  thus 


<  IF'<x  )  -  F’(x)5!w  -  x1>  , 


d[w,X.}  <  X(X.)Eqi  +  X(x  )iF'(x  )  -  F*(x)Uw  -  X  i 


<  X(XQ)lqS  +  —  ajw.Xp]  . 


Therefore 
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dfw,Xnj  <  2X{xQ}Iqi  , 

and  so 

{?  A  X  )(c)  C  (T  X  HO)  *  2X(x  }IqIS  , 
x  e  x  £  0 

which  proves  that  A  is  upper  Lipschitziar.  or.  ^  q ( x  , ! ?.  with  modulus  2?. <x 

elementary  compactness  argument  now  shows  that  for  some  X  and  some  -  >  0,  for 

x  e  Xp,  ?x  n  is  upper  Lipschitziar.  on  nB  with  modulus  X. 

Choose  a  positive  6  <  nin{c,n}  so  that  if  S  <  o  then  ,sax{  1  ,X}3C  5) 

q  e  r)B  and  let  x  f  (K  1  n  X.Mq).  Then  q  £  F(x)  -  ?.{x),  so  if  x.  f  Xp  wi 

lx  -  x^E  =  d[x,Xe]  we  have 

q  +  LF  (x)  -  F(x)  e  LF  (x)  *  P.Cx)  , 

X1  X1 


and  therefore  x  e  (T(x  )  Ax  ) [q  +  LF  (x)  -  F(x)>.  However, 

£ 


Eq  +  LFX  (x)  -  F(x)  S  <  Iql  +  Sdx  -  x^JIx  -  Xl 


<  \  n  +  \  n  <  n 


so  by  upper  Lipschitz  continuity  of  T C x ^ )  n  X£, 

d[Xi,x0]  <  xiqi  |  atvV 

implying 

dJx^Xg]  <  2X Iqi  . 

Hence 

(h”1  a  X-Ho)  C  X.  +  2XlclB 
0  0 

for  each  a  e  nB. 

How  let  Xq  e  Xp.  By  hypothesis 

0  c  (LFx  +  R) (Xp)  =  F(x0>  +  R(x0)  =  H(x0)  , 

Xp 

_1 

so  Xp  C  (H  ’  A  XgHO).  On  the  other  hand,  by  applying  (4.1)  with  q  =  0  we  see 
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(H-1  '  X.  )(0)  C  x  ,  so  that  (H  1 


X.)(  0)  =  X,  and  H 


—  n3  with  modulus  2X.  This  completes  the  proof. 

This  result  will  allow  us  to  measure  the  displacement  in  a  stationary  p 
multipliers  u  in  terms  of  perturbations  in  the  problem  functions  and  their 
Ste  show  this  next,  and  then  proceed  to  develop  a  simpler  measure  of  displace: 
case  in  which  the  functions  involved  are  Lipschitziar.  in  p.  In  what  follow 
Uq  for  U(x0,pq). 

THEOREM  4.2:  Assume  the  hypotheses  of  Theorem  3.3,  and  suppose  also  th, 
are  polyhedral.  Then  there  are  a  neighborhood  '.'--j  cf_  pn  and  a  constant 
for  each  p  r  ”T,  each  x  e  SP(p)  and  each  u  e  UCx.p), 

d[{x,u),{xR}  x  <  ulF(x,U,pQ)  -  F(x,u,p)S  , 


'  f(x,p)  +  g1  (x,p)*u  "1 

F(x,u,p)  := 

-gCx.p)  J 


PROOF:  Ve  shall  apply  Theorem  4.1  to  the  function  F(x,u)  given  by  FCx,u,p5).  Ke 
take  R  to  be  and  as  a  candidate  for  Xn  we  take  the  set  {xR  j  x  U^.  Because  of 

the  special  form  of  X0,  it  is  clear  that  F*  is  continuous  there:  the  rest  of  hypothesis 
(iii)  follows  from  the  differentiability  assumptions  and  the  fact  that  Un  is  bounded. 

For  (i),  we  have  to  show  that  if  un,u1 ,u2  €  Oq  then 

LF,  ,(u_)  =  IF  ,(u,)  . 

(VV  2  Vi*  2 


However,  an  algebraic  computation  shows  that  each  of  these  quantities  is  equal  to 


C'(VV  + 

-9(VP05 


so  that  (i)  is  satisfied.  For  hypothesis  (ii),  we  suppose  that  U0  f  I)q  and  that  for 
some  x  and  u,  0  €  (LF^  u  )  +  ®’’cxQ^X,U^*  x  is  a  stationary  point,  and  u  is 

an  associated  multiplier,  for  the  quadratic  programming  problem  Ccf.  (2.2)); 


Let  C  =  R2,  Q  =  R^,  and  P  =  { <P,  »P2  >  f  R  10  <  P,  <  1,  I?,'  <  J  p^>.  Define 

f(x,p)  ^  lx(" 

and 

g(x,p)  :=  -(A  +  p1I)x  +  ^  (2  +  p  )a  +  p„b  , 

where 


It  is  easy  to  check  that  the  hj  theses  of  Corollary  4.3  are  satisfied.  However,  for 
any  p  e  P  the  unique  minimizer  oi  the  problem  (3.1)  with  the  above  data  is  given  by 

a  if  p  -  (0,0) 

l  a  +  p"1p2b  if  p  *  (0,0)  , 

and  x( » )  is  obviously  not  locally  Lipschitzian  at  (0,0),  although  it  does  satisfy  the 
bound  (4.4).  This  example  also  shows  that  the  continuity  result  of  Kojima  ilO,  Th.  7.2] 
for  the  case  of  regular  constraints  cannot  be  strengthened  to  prove  Lipschitz  continuity. 
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